Image sensors are increasingly being used in conjunction with displays to enable many new applications. Examples of these application include (a) video conferencing, where the user transmits an image of itself while looking at the image of a remote user, (b) user interfaces, where the user gesticulates as a way of controlling the display contents (e.g. changing channels in a TV), (c) head tracking, where the position of the head is used to modify the display content in a computer user interface, or for stereo viewing in auto-stereoscopic displays, and (d) environmental control, where the display might change the contents or its operation in response to changes on the ambient illumination, viewer position and number of viewers.
In one traditional implementation, an image sensor has been placed in close proximity to the edge of the display, usually in its bezel, at the top. Unfortunately, there are many limitations resulting from this placement of the image sensor. For example, the usefulness of such camera/image sensor systems is limited by the excessive parallax when the user viewing position is close to the display surface, as in a desktop computer display, or when the displays are very large.
In another traditional implementation, telepresence systems have been customized to include large displays providing real size images of participants, and are placed far from the users to minimize the parallax. Unfortunately, other factors that greatly influence the quality of the teleconference are the delay, and image quality including resolution and color matching. In yet another implementation, to achieve eye-to-eye communication, several devices have been proposed and described, including the use of beam splitters in a tele-prompter like configuration, or embedding of the camera behind the display.
Several limitations are associated with the aforementioned implementations traditionally utilized, including 1) Fixed Point of View (POV)—the image of the local user can only be captured from one point-of-view, fixing also the placement of the image of the remote user on the display to coincide with the camera POV; 2) Fixed Field of View (FOV)—the view is fixed because the camera or cameras behind the display cannot change focal length with respect to the local user position—this restricts the user to a single correct viewing position, with the image becoming increasingly distorted away from it; and 3).
Bulk size—the size of a typical high quality camera, and the FOV requirement for a natural image, will require the camera to be removed back from the viewing surface, greatly increasing the bulk of the display device and making any such method impractical for use in laptop displays, cell phones and other apparatus where the thickness of the display should remain small.
There is thus a need for addressing these and/or other issues associated with the prior art.